Experiments in Data Format Interoperation Using Defuddle

This document discusses the status of the Defuddle parser and recent work conducted as part of the “Innovative Systems and Software: Applications to NARA Research Problems” project. Robust sharing, reuse, and curation of data requires a clean separation of issues related to bits, formats, and logical content. To address these issues the Open Grid Forum is defining the Data Format Description Language (DFDL) standard for describing the structure of binary and textual files and data streams so that their format, structure, and metadata can be exposed as XML [3]. While this is sufficient for describing the internal layout of data (the “syntax”), interoperability and curation also require description of logical relationships within and between data sets in terms of globally understood concepts (the “semantics”). Extending the concept of the DFDL, and the Defuddle DFDL parser implementation, we have defined a two-step declarative mechanism for describing the structure and relations in binary and ASCII data in terms of vocabularies defined using standard Semantic Web languages (RDF and OWL). We briefly outline our approach below and highlight the potential benefits of exposing internal data semantics in the context of larger semantic systems. DFDL and Defuddle separate the issues: • The DFDL annotate schema defines the logical structure of the data • The DFDL annotations map the bits to the logical structure • The XML output of Defuddle provides a standard format with a well-defined logical structure • The RDF output of Defuddle provides description of logical relationships in a standard format This document describes two demonstrations of data interoperation with Defuddle. These demonstrations explore how Defuddle and DFDL can be used for tasks that usually are implemented with code. The first demonstration explores an aspect of file characterization, namely recognition of the “MIME-type” of a file. The second demonstration illustrates a simple example of recognition and 3D data format interoperation. In both these cases, the Defuddle parser software was used without modification. Each application required development of appropriate DFDL-annotated XML schemas to read the binary or text data. Also, the MIME-type recognizer used the semantic extensions to produce an RDF triple that asserts the MIME-type. The rest of the paper is organized as follows. Section 2 gives some background on the Defuddle technology. Section 3 explains the file identification demonstration. Section 4 discusses the 3D formats demonstration. Section 5 concludes. Appendices give details of the XML schemas and XSL transformst used. Data Interoperability Experiments